Identifying Misconceptions about Confidence Intervals

Although confidence intervals (CIs) have many benefits over null hypothesis significance testing (NHST) they can still be misinterpreted. Identifying CI misconceptions is a first step in designing teaching tools that can be used to prevent or reduce them. I surveyed graduate level students and found they hold several misconceptions about CIs. Many believe there is a uniform likelihood distribution across a CI, with a high proportion of these showing a cliff effect (a sudden major drop in likelihood at each limit of a CI). Many students also misunderstand the relationship between the width of a CI and the confidence level. In this paper I present a taxonomy of CI misconceptions identified by empirical studies, and explore faulty conceptual models that may be the source of the misconceptions. I also propose an educational tool that could be used to confront CI misconceptions, particularly misconceptions about CI distributions.